On the Differential Privacy of Bayesian Inference

We study how to communicate findings of Bayesian inference to third parties, while preserving the strong guarantee of differential privacy. Our main contributions are four different algorithms for private Bayesian inference on proba-bilistic graphical models. These include two mechanisms for adding noise to the Bayesian updates, either directly to the posterior parameters, or to their Fourier transform so as to preserve update consistency. We also utilise a recently introduced posterior sampling mechanism, for which we prove bounds for the specific but general case of discrete Bayesian networks; and we introduce a maximum-a-posteriori private mechanism. Our analysis includes utility and privacy bounds, with a novel focus on the influence of graph structure on privacy. Worked examples and experiments with Bayesian na{\"i}ve Bayes and Bayesian linear regression illustrate the application of our mechanisms.Comment: AAAI 2016, Feb 2016, Phoenix, Arizona, United State

Asymptotic energy of graphs

The energy of a simple graph $G$ arising in chemical physics, denoted by $\mathcal E(G)$, is defined as the sum of the absolute values of eigenvalues of $G$. We consider the asymptotic energy per vertex (say asymptotic energy) for lattice systems. In general for a type of lattice in statistical physics, to compute the asymptotic energy with toroidal, cylindrical, Mobius-band, Klein-bottle, and free boundary conditions are different tasks with different hardness. In this paper, we show that if $\{G_n\}$ is a sequence of finite simple graphs with bounded average degree and $\{G_n'\}$ a sequence of spanning subgraphs of $\{G_n\}$ such that almost all vertices of $G_n$ and $G_n'$ have the same degrees, then $G_n$ and $G_n'$ have the same asymptotic energy. Thus, for each type of lattices with toroidal, cylindrical, Mobius-band, Klein-bottle, and free boundary conditions, we have the same asymptotic energy. As applications, we obtain the asymptotic formulae of energies per vertex of the triangular, $3^3.4^2$, and hexagonal lattices with toroidal, cylindrical, Mobius-band, Klein-bottle, and free boundary conditions simultaneously.Comment: 15 pages, 3 figure

Analysis of networks: privacy in Bayesian networks and problems in lattice models

© 2016 Dr Zuhe ZhangThis thesis deals with differential privacy in Bayesian inference, probabilistic graphical models and information-theoretic settings. It also studies the expansion property and enumeration problems of certain subgraphs of networks. The contributions of this thesis fall into three main categories: (i) We establish results for Bayesian inference, providing a posterior sampling algorithm preserving differential privacy by placing natural conditions on the priors. We prove bounds on the sensitivity of the posterior to training data, which delivers a measure of robustness, from which differential privacy follows within a decision-theoretic framework. We provide bounds on the mechanism's utility and on the distinguishability of datasets. These bounds are complemented by a novel application of Le Cam's method to obtain lower bounds. We also explore inference on probabilistic graphical models specifically, in terms of graph structure. We show how the posterior sampling mechanism lifts to probabilistic graphical models and bound KL-divergence when releasing an empirical posterior based on a modified prior. We develop an alternate approach that uses the Laplace mechanism to perturb posterior parameterisations, and we apply techniques for released marginal tables that maintain consistency in addition to privacy, by adding Laplace noise in the Fourier domain. We also propose a maximum a posteriori estimator that leverages the exponential mechanism. (ii) We generalize a prior work that considered differential privacy as a trade-off between information leakage and utility in noisy channels. By assuming certain symmetric properties of the graphs induced by the Hamming-1 adjacency relation on datasets, the authors showed the relation between utility and differential privacy. We prove the utility results still hold without any assumption on the structure of induced graphs. Our analysis applies to the graph of datasets induced by any symmetric relation, therefore is applicable to generalized notions of differential privacy. (iii) In a different direction in graph analysis within statistical mechanics, we discover the relation between graph energy per vertex of a regular lattice and that of its clique-inserted lattice using spectral techniques. We obtain the asymptotic energy per vertex of 3-12-12 and 3-6-24 lattices. We derive the formulae expressing the number of spanning trees and dimer covering of the k-th iterated clique-inserted lattices in terms of those of the original one. We show that new families of expander networks can be constructed from the known ones by clique-insertion. We modify the transfer matrix method and use it to obtain upper and lower bound for the entropy of number independent sets on the 4-8-8 lattice. We show that the boundary conditions have no effect on the entropy constant. We also introduce a random graph model, where we study the annealed entropy of independent set per vertex. We show that the annealed entropy can be computed in terms of the largest eigenvalue (in modulus) of corresponding expected transfer matrix. Experiments suggest that this annealed entropy is highly correlated to the corresponding Shannon entropy

On the Differential Privacy of Bayesian Inference

International audienceWe study how to communicate findings of Bayesian inference to third parties, while preserving the strong guarantee of differential privacy. Our main contributions are four different algorithms for private Bayesian inference on proba-bilistic graphical models. These include two mechanisms for adding noise to the Bayesian updates, either directly to the posterior parameters, or to their Fourier transform so as to preserve update consistency. We also utilise a recently introduced posterior sampling mechanism, for which we prove bounds for the specific but general case of discrete Bayesian networks; and we introduce a maximum-a-posteriori private mechanism. Our analysis includes utility and privacy bounds, with a novel focus on the influence of graph structure on privacy. Worked examples and experiments with Bayesian naïve Bayes and Bayesian linear regression illustrate the application of our mechanisms

RGBD Video Based Human Hand Trajectory Tracking and Gesture Recognition System

The task of human hand trajectory tracking and gesture trajectory recognition based on synchronized color and depth video is considered. Toward this end, in the facet of hand tracking, a joint observation model with the hand cues of skin saliency, motion and depth is integrated into particle filter in order to move particles to local peak in the likelihood. The proposed hand tracking method, namely, salient skin, motion, and depth based particle filter (SSMD-PF), is capable of improving the tracking accuracy considerably, in the context of the signer performing the gesture toward the camera device and in front of moving, cluttered backgrounds. In the facet of gesture recognition, a shape-order context descriptor on the basis of shape context is introduced, which can describe the gesture in spatiotemporal domain. The efficient shape-order context descriptor can reveal the shape relationship and embed gesture sequence order information into descriptor. Moreover, the shape-order context leads to a robust score for gesture invariant. Our approach is complemented with experimental results on the settings of the challenging hand-signed digits datasets and American sign language dataset, which corroborate the performance of the novel techniques